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(-^ I Abstract 

, This paper considers compressed sensing matrices and neighbor- 

liness of a centrally symmetric convex polytope generated by vectors 
zbXi, . . . , ±X]\[ E R", (A^ > n). We introduce a class of random sam- 
pling matrices and show that they satisfy a restricted isometry prop- 
J> I erty (RIP) with overwhelming probability. In particular, we prove 

that matrices with i.i.d. centered and variance 1 entries that satisfy 
. uniformly a sub-exponential tail inequality possess this property RIP 

'n^ I with overwhelming probability. We show that such "sensing" matri- 

Tlj" ■ ces are valid for the exact reconstruction process of m-sparse vectors 

. via £i minimization with m < Cn/ {cN / n) . The class of sam- 

\ pling matrices we study includes the case of matrices with columns 

that are independent isotropic vectors with log-concave densities. We 
deduce that \l K d is a convex body and Xi, . . . ,X7v G K are 
i.i.d. random vectors uniformly distributed on then, with over- 
I whelming probability, the symmetric convex hull of these points is an 

m-centrally-neighborly polytope with m ~ n/ log^{cN/n). 
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1 Introduction 



Let l<m<n<A^be integers and let Xi, . . . , G M". Denote by A the 
n X N matrix with Xi, . . . , Xn as columns and by K{A) = K{Xi, . . . , X^) 
the convex hull of ±Xi, . . . , ±Xn. Recall that a centrally symmetric convex 
polytope is m-centrally-neighborly if any set of less than m vertices con- 
taining no-opposite pairs, is the vertex set of a face (see the books \T6\ and 

The connection between the neighborliness of K{A) and sparse solutions 
of underdetermined linear equations was discovered in [11], Theorem 1, where 
it is proved that the following two statements are equivalent: 

i) K{A) has 2N vertices and is m-neighborly 

ii) whenever y = Az has a solution z having at most m non-zero coordi- 
nates (in other words z is m-sparse), then z is the unique solution of 
the program: 

(P) minlltll^i, At = Az. 

Here the £i-norm is defined by = ^Yl!i=\ ^'^y ^ — ^i)f=\ ^ 

Statement i%) is the so-called exact reconstruction problem by l\ mini- 
mization or hasis pursuit algorithm. For a more detailed and complete anal- 
ysis of the reconstruction of sparse vectors by the basis pursuit algorithm we 
refer to [7] and [12]. 

Let us also mention in the same stream of ideas that problem ii) is dual 
to the problem of decoding by linear programming. In this latter problem a 
linear code is given by the matrix A*, and thus a vector x G M" generates 
the vector A*x E defined by measurements ((Xi, x), . . . , {Xjy, x)) . Sup- 
pose that A*x is corrupted by a noise vector z G which is assumed to 
be m-sparse. The problem is to reconstruct x from the data, which is the 
noisy output y = A*x + z. This problem is then tackled by a linear program- 
ming approach (see [S] for complete references) that consists of the following 
minimization problem 

(P') mm\\y - A*th,. 

Let us denote by | ■ | the natural Euclidean norm in and M^. Looking 
for a sufficient condition for a given matrix M to satisfy condition ii), the 
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authors of [S] introduced the concept of Restricted Isometry Property (RIP) 
defined by the following parameter. 

Definition. Let M be a n x N matrix. For any 1 < m < min(n, A^), the 
isometry constant of M is defined as the smallest number 6m = 6m{M) so 
that 

{l-6mM<\Mz\^<il + 6mM 

holds for all m-sparse vectors z G M^. The matrix M is said to satisfy the 
Restricted Isometry Property of order m with parameter 6, shortly KIP m{6), 
z/0 < 6miM) < 6. 

The relevance of this parameter for the reconstruction property ii) is for 
instance revealed in [7],[S], where it was shown that if 6m{M) + 62m{M) + 
63m{M) < 1 then M satisfies ii) (see also [6J, [9J, [18j). In the present paper, 
we shall use the following sufficient condition from [5]: if a matrix M satisfies 

62m {M)<V2-1 

then i) and ii) are satisfied. In other words, if M has RIP2m("\/2 — 1) then 
M has the reconstruction property ii). This approach gives the strategy of 
our paper. 

Recall that no general construction of centrally symmetric polytopes is 
known to produce polytopes with an optimal order of neighborliness. All 
known results are of randomized nature, namely, they show that for a certain 
probability on the space of nx N matrices, a polytope K[A) is m-neighborly 
with overwhelming probability, for (large) m depending on n and N. Con- 
sequently, from now on, A will be a random matrix in some Ensemble in the 
sense of Random Matrix Theory. Due to the normalization, we shall consider 
the isometry constant of Aj ^Jn. The plan consists in specializing to some 

model of random matrices, the condition 62m (^^) < ~ ^■ 

Let Xi, . . . , G M" be independent random vectors normalized so that 
E|Xjp = n for alH = 1, . . . , A^. The model we will develop here is structured 
by two conditions: an inequality of the tails of linear forms and an inequality 
of concentration of the Euclidean norm. 

• Linear forms obey a uniform sub-exponential decay, that is, for all 
1 < z < A^, all y e and t > 0, 

F{\{X„y)\>t) <Cexp{-ct), 

where C,c> 0. 



3 



• The Euclidean norms of Xi, . . . , are concentrated around their av- 



Note that such a concentration inequahty is clearly necessary in order to have 



One of the main results of this paper, Theorem 14.31 claims that under 
these conditions, whenever 



the random polytope K{A) is m-centrally-neighborly with probability larger 
than 1 — 2X — C exp(— c-^/n), where C,c> are universal numerical constants. 
We will make it more precise in Section HI This model includes the cases when 

• Xj's are independent isotropic random vectors with a log-concave den- 



• the entries of the matrix are independent, centered with variance one 
and satisfy a sub-exponential tail inequality; 

• Xj's are on the sphere of radius ^/n and linear forms exhibit a uniform 
sub-exponential tail inequality. 

These examples give rise to new classes of compressed sensing matrices. The 
class of i.i.d. entries with sub-exponential tail behavior (that is, entries being 
tpi random variables), contains a subclass of matrices with i.i.d. ipr entries 
for 1 < r < 2 (see Definition 12. II below of ipr random variables). Since in this 
case the obtained bounds are better by a power of logarithm that may be 
essential in applications, we prove our results in full generality, for 1 < r < 2. 

Sub-gaussian matrices with independent ip2 entries, which correspond to 
r = 2, are by now well understood. They include for instance the Gaussian 
case when the matrix A is built with i.i.d. Gaussian A^(0, 1) random variables 
(see [12] , [S] , ) ; the case when the entries of A are i.i.d. (±1) Bernoulli 
random variables ([5], [23], [3]); a general case of i.i.d. sub-gaussian entries 
is treated in ([23],[2l], also see [25] for simpler proofs). 

Results of this paper are based on concentration type inequalities for 
random matrices under consideration. The proof of the main technical result, 



erage: 




RIPi((v^-l)/2). 



m < cn/ \og^{CN/n), 



sity; 
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Theorem 13 .21 will employ methods from pQ. A crucial new ingredient consists 
of an analysis of the quantity 



sup 



i<N 



-E 



i<N 



1/2 



where Um denotes the set of norm one m-sparse vectors in M^. In Section [2] 
we present some definitions and preliminary tools. In Section [3] we apply 
Theorem 13.21 to estimate the isometry constant (Theorem 13. 3p . Then we 
study the m-neighborly property of random polytopes in Section |4] and give 
application to polytopes generated by random points from a convex body, 
polytopes generated by independent vectors with independent ipr random 
coordinates, and polytopes generated by independent ipr random vectors on 
a sphere. Section [5] is devoted to the proof of Theorem 13.21 and discussion of 
optimahty of the result. 



2 Notation and preliminaries 

We equip M" and with the natural scalar product ( ■, ■) and the natural 
Euclidean norm | ■ | . We use the same notation | ■ | to denote the cardinality 
of a set. Unless otherwise stated, (Xj)j>i will denote independent random 
vectors in M". By ||M|| we shall denote the operator norm of a matrix M, 
that is, ||M|| = sup|y|^;^ \My\. 

Definition 2.1. For a random variable F G M and r > we define the 
ipr-norm by 

\\Y\\^^ = inf > ; Eexp {\Y\/Cy < 2} . 

It is well known that the ipr-norm of a random variable may be estimated 
from the growth of the moments. More precisely if a random variable Y is 
such that for any p > I, \\Y\\p < p^^^K, for some K > 0, then ^ c/T 

where c > is a numerical constant. 

Definition 2.2. Let X G M" be a centered random vector and r > 0. We 
say that X is ipr or a ipr vector, if sup^g^n-i || {X,y) \\^^ is bounded and we 
set 

\\XU, = sup II {X,y) 11^^. 
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Remark: The above notation of for the weak ipr norm of a random 

vector X should not be confused with the standard convention in the proba- 
bihty theory that this notation stands for the ipr norm of the random variable 
|X|, i.e., II |X| ll^^-this latter meaning will never be used in this paper. 

We recall the well known Bernstein's inequality which we shall use in the 
form of a ipi estimate ([30]). 

Lemma 2.3. Let Yi,...,Yn be independent real random variables with zero 
mean such that for some ip > and every i, \\Yi\\^^ < ip. Then, for any 
t > 0, 

Given a set C {1,...,A^} by Pe we denote the orthogonal projection 
from onto the coordinate subspace of vectors whose supports are in E. 
We denote this subspace by M^. The support of 2 e is denoted by supp z. 
A vector 2 e is called m-sparse if | supp 2; | < m. The subset of m-sparse 
unit vectors in is denoted by 

Um = Um{^^) ■■= {z eR^ : \z\ = 1, I supp^l < m}. (2.1) 

Let = {x = (xj) G : ||a;||oo = maxj |xj| < 1} and be the 
unit Euclidean ball. For every E C {1, A^}, e,a E (0, 1] by J\f{E, e, a) we 
denote an 5-net in B2 H aB^ fl M.^ in the Euclidean metric. Thus for every 
X G B2 n aB^ supported by E, there exist x G J^{E, e, a) supported by E 
such that |a; — a;| < e. A standard volume comparison argument shows that 
we may assume that the cardinality of Af{E,e,a) does not exceed (3/e)™, 
where m is the cardinality of E. 

Definition 2.4. A random vector X G M" called isotropic if 

E{X,y) = 0, E|(X,?/)p = |yp forallyGM", (2.2) 
in other words, if X is centered and its covariance matrix is the identity. 

A subset C M" is said to be isotropic when a random point X uniformly 
distributed in K is an isotropic random vector. 

Recall that a function / : M" M is called log-concave if for any 6 G [0, 1] 

and any Xi,X2 G M", 

f{ex^ + ii-e)x2)>fix^Yfix2y-'. 
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It is well known [1] that if a measure has a log-concave density, then linear 
functionals exhibit a sub-exponential decay. More precisely, we have: 

Lemma 2.5. Let X G M" 6e a centered random vector with a log- concave 
density. Then for every y G S*""^, 

II {X,y) U,<c{E\{X,y)\Y\ 

where c > is a universal constant. As a consequence, if X is an isotropic 
random vector with a log-concave density then \\X\\^^ < c. 

The Euclidean norm of an isotropic random vector with a log-concave 
density highly concentrates around its expectation, this translates geometri- 
cally to the concentration of mass of an isotropic convex body within a thin 
Euclidean shell ([20], see also [T5j). We will use here the following result 
immediately derived from [19], Theorem 4.4. 



Lemma 2.6. Let 1 < n < N be integers and let Xi, . . . , Xn G be 

isotropic random vectors with log- concave densities. There exist numeri- 
cal positive constants C, cq and ci G (0, |) such that for all 9 G (0, 1) and 
N < exp(c^'=ora'=i), 



P I max 

i<N 



IX,; 



1 



>9)< Cexp{-cd^''n 



n 

Moreover, one can take cq = 3.33 and c\ = 0.33. 

Remark: It is conjectured that in the above theorem one can replace 

^3.33^0.33 by c{e)n^/\ 

We shall also use the following result from p6] as formulated in [1]. 

Lemma 2.7. Let N,n > 1 be integers and let Xi, . . . , Xn G M" be isotropic 
random vectors with log-concave densities. Then there exists an absolute 
positive constant Cq such that for any N < exp{^/n) and for every K > 1 
one has 

max IXA < CqK^/u 

i<N 

with probability at least 1 — exp(— i^y^). 

In this paper, different universal positive constants may be denoted by 
the same letters C, Cq, C", c, Cq, c', etc. 
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3 Isometry constant 



We begin this section by formulating, in Theorem I3.3[ a general estimate 
for the isometry constant of random matrices with independent ipr columns. 
Then, in order to apply such an estimate, we introduce two sufficient con- 
ditions that determine large classes of random matrices. Finally, we give 
examples of important classes that satisfy the estimates from Theorem 13.31 
and thus provide us with models: the Log-Concave Ensemble, matrices with 
i.i.d. ipr entries, and matrices defined by independent ipr vectors on a sphere. 



3.1 Estimating the isometry constant 

Techniques of "compressed sensing" rely on properties of the sampling ma- 
trix, which should act almost isometrically on sparse vectors. This motivated 
the concept of Restricted Isometry Property (RIP) defined in [8]. To quantify 
this property of the "sensing" matrix, the authors introduced the isometry 
constant defined in the introduction, that we recall here for the convenience 
of the reader. 

Definition 3.1. Let M be a n x N matrices and let 6 G (0,1). For any 
1 <m < min(n, A^), the isometry constant of M is defined as the smallest 
number 6^ = Sm{M) so that 



:i-5™)kr < \Mz\^ < ii+5m)\z\ 



(3.1) 



holds for all m-sparse vectors z G R^. The matrix M is said to satisfy the 
Restricted Isometry Property of order m with parameter 6 ifO< 6m{M) < 6. 

Let Xi, . . . , Xat e M" and let A = A^"'^) be the "sampling" matrix with 
the Xj's as columns. We begin by a simple observation. Define the following 
quantity 



sup 

Z&Um 



i<N 



i<N 



1/2 



(3.2) 



Then, clearly 



sup 



n 



< — + max 

n i<N 



- 1 



n 



(3.3) 
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Thus the isometry constant is controlled by quantity and the second 



term, max 



■i<N 



. We begin by estimating B„ 



Theorem 3.2. Let n > 1 and 1 < m < N be integers. Let 1 < r < 2 and 

Xi, . . . , Xn G be independent ipr random vectors with ip = maxi<7v \\X,, 
Let 9 E (0, 1/4), K,K' > 1 and assume that m satisfies 



i\\lpr 



2N 
6m 



m \o^^^ - — < (P'n. 



Then setting ^ = ipK + K' , the inequality 

El < CeOn 

holds with probability at least 



1 — exp ( —cK^^/m\og 



-P 

Om) ) V 



max IXjl > K' \fn 



where C, c are absolute positive constants. 



We postpone the proof of Theorem 13.21 to the last section. Combining 
this theorem with inequality (13. 3p . relating the RIP, and concentration 
of the Euclidean norm of the Xj's, we immediately deduce an estimate for 
the isometry constant of a random matrix with independent ipr columns. 

Theorem 3.3. Let n > 1 and m, N be integers such that 1 < m < min(iV, n) . 
Let 1 < r < 2. Let Xi, . . . , X^q G M"" be independent ipj. random vectors and 
let ^ = maxi<7v Let 9' G (0, 1), K,K' > 1 and set ^ = ijjK + K' . 

Then 

~ ' eN 




holds with probability larger than 

1 — C exp —cK^ \/m\og 



- P ( maxlXil > K 
where C, c > are universal constants. 




> 9' 
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Note that such an estimate for 6m i^) is meaningful only if, firstly, its 




right hand side is < 1, and secondly, if it holds with probability > 0. In 
fact, the former condition is equivalent to the RIP of order m. This leads to 
considerations of models of random nx N matrices that satisfy the following 
two conditions. Let 1 < r < 2, > and A G (0, 1). Let Xi, . . . , X^, G R"- 
be independent ipr random vectors and let A be the matrix with Xi, . . . , 
as columns. 

• Condition Hi{r,il)): Linear forms obey a uniform ip^ estimate: 



• Condition H2{X): |Xj|'s are concentrated around their average: 



As already mentioned in the Introduction, a condition such as H2{X) is 
necessary to have the RIP. Indeed, if the matrix A/ ^Jn has RIPi((-\/2 — l)/2) 
with probability A then H2{X) is satisfied. 

3.2 Examples 

We now specialize Theorem 13.31 to some specific classes of matrices. 
3.2.1 The Log-Concave Ensemble 

We start by considering the "log-concave setting", where Xi, . . . ,Xi^ G 
are independent isotropic vectors with log-concave densities. 

Lemma 3.4. Assume the above "log-concave setting". There exist universal 
constants il!,C,c > such that conditions Hi{1,iIj) and H2{C exp{—cn'^^)) 
are satisfied whenever N < exp {cn'^^), where C\ is given in Lemma \2. 61 

The proof is immediate from Lemmas 12.51 and 12.61 

Applying Theorem 13.31 (with r = 1) together with Lemmas 13.41 and 12.71 to 
the Log-Concave Ensemble, we get that for every X < exp(cr;,'^^). 



< ^ for all y G 5""^ and 1 < z < X. 



(3.4) 




(3.5) 




) 



+ 



V2-I 



2 



(3.6) 
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holds with probabihty larger than 



1 — C exp —ci/m log 




) 



) 



e-'^-exp(-cn'=i) 



where C,c> are universal constants and Ci is given in Lemma 12.61 

It might be worthwhile to note that using directly Lemma 12.61 one can 
replace the second term in estimate (13. 6p by a term tending to when 
n — cx), but this would require an adjustment in probability. For example 
]^^^ci/2co ^orks with the probability estimate in which exp(— cn'^^) is replaced 
by exp(— cn'^^/^). (Here cq is given in Lemma [2. 6[ ) 

3.2.2 Matrices with independent ipr entries 

Consider now the ^^ipr setting", where the entries of the matrix A are 
independent centered, with variance one, random tpr variables (with r G 
[1,2]). Set ip = maxy ||aij||v.,- 

Lemma 3.5. Assume the above "ipr setting" with r G [1,2]. Then con- 
ditions Hi(r,C-ip) and if2(2 exp(— cn'"/^/-?/'^'")) are satisfied whenever N < 
exp(cnJ'^'^ /'ijj'^^), where C,c are absolute positive constants. 

Proof. To prove that the columns of the matrix A are ipr vectors we will 
estimate the p-th moments of random variables Y17=i Vi^^iji ^i^y H = {Hi) ^ 

and any p > 1. This will be done by using Talagrand's concentration 
inequality for linear combinations of symmetric WeibuU variables together 
with some symmetrization and truncation arguments. 

The following Lemma is a combination of Corollaries 2.9 and 2.10 of [25] . 

Lemma 3.6. Let r G [1,2] and Yi, . . . , 1^ be independent symmetric random 
variables satisfying P(|l^,| > t) = exp(— f). Then for every vector a = 
(oi, . . . , a^i) G and every t >0, 



where 1/r* + 1/r = 1 and \\a\\q = {\ai\'^ + . . . + \an\'^y^'', for 1 <q < oo. 
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The behavior of general centered ipr variables can be easily reduced to 
symmetric WeibuU variables. The argument is quite standard, we sketch it 
here for the sake of completeness. 

Assume thus that Zi, . . . , Z„ are independent mean zero random variables 
with < 1. Let /3 = (log2)i/^ and set Ui = {\Zi\-l3) + . Let Yi be defined 

as in Lemma 13. 6[ 

We have for t > 0, 

¥{Ui >t)<¥{\Z,\>t + (3)<2 exp(-(t + 
<2exp{-f - (3') =P(|F,| >t). 

We will use the above observation together with symmetrization and the 
contraction principle to estimate moments of linear combinations of variables 
Zj. We have for p > 1, 



2^ciiZi < 211 (symmetrization) 

1=1 

n 

= 2 ^ ^ aj5j|Zj 



i=l 



i=l 



< 2 2,'^«^j(/5 + (the contraction principle) 

II ^ lip 

1=1 

n n 

<2\y^aieil3\ +2^ajeif/j 
II Ip II 

1=1 i=l 

n 

<Cv^/9||«||2 + 2||5^a,£,rJ| 



i=l 



< C^\\a\\2 + Cp^'''\\a\\r*, 

where to get the last two inequalities we used Khinchine's inequality, Lemma 
13.61 and integration by parts to pass from tail to moment estimates. 

We are now ready to prove condition Hi{r,Cip). Fix y G S"~^ and 
consider the linear combination Yll=iyi^ij- Since < ip, we obtain by 

homogeneity 



n 
i=l 
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since r G [1,2] implies that p^f"^ > y/p and ||a||r* < ||a||2 = 1- The growth 
condition on the moments of the random variable Y17=i Vi^ij iniphes that its 
ipr norm is bounded hy Ctp. 

The proof of condition H2 goes along similar lines. Instead of Lemma [3?6] 
we will now use the following lemma, which is an easy consequence of The- 
orem 6.2 in [T7] and the observation that the p-th moment of a WeibuU 
variable with parameter s is of order CsP^^^, where Cg remains bounded for 
s away from 0. 

Lemma 3.7. IfO < s < 1 and Yi, . . . ,Yn are independent symmetric random 
variables satisfying P(|Vj| > t) = exp(— t*), then for a = (ai, . . . , a„) G 
and p > 2, 

n 

l/s 



1=1 



\a\\p. 



Moreover, for s > 1/2, Cs is bounded by some absolute constant. 

Using similar arguments as in the proof of condition Hi we can infer 
from the above lemma that if Zi , . . . , Z„ are independent mean zero random 
variables with ||Zi||^^ < 6 (s G [1/2, 1)), then for p > 2, 



Va,Z, <Cbi^\\a\\2+p'^ 
Therefore, for any p > 2 by the Chebyshev inequality in Lp, 

n 

P(^| > Cb{^ + p^''n^/'P)^ < exp(-p). 



4 = 1 



For p > 3 we have 

with C universal for s > 1/2, so the above inequality yields 



p(|^Zi| > + <e^exp(-p) 

i=l 

for some (new) universal constant C or equivalently 

> t) < 2exp (-cmin [— , [-J]). (3.7) 



2=1 
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For fixed j we apply this inequality with s = r/2 to variables Zi = of- — 1 . 
Note that EZ, = and 

= C{l + \\a,,\\l;)<C^\ 

(The additional constants appearing above stem from the fact that under 
the standard definition for s < 1, || • ||^^ is not a norm but only a quasi-norm 
and additionally 7^ 1- O^^^ can modify the function x ^— e^"^ — 1 so 

that it is convex. For r away from zero, this modification changes the norm 
by an absolute constant). Therefore, applying (13. 7p with t = en yields 

KkE4-l|£-)£2exp(-cmin[lJ,(p)' ]) 

<2exp(-g^j. 

For r = 2 the proof is similar, but uses Lemma 13.61 (which in this case 
reduces to Bernstein's ipi inequality) instead of Lemma 13.71 (the argument is 
simpler since in this case the involved norms of the vector a do not depend 
on p and we get (13. 7p directly) . 

The lemma follows now by the union bound. □ 

Applying Theorem 13.31 together with Lemma 13.51 to the "^/^^ setting" , we 
get that for every < exp(cn'"/^/'?/'^'"), 

holds with probability at least 

1 -Cexp(-cn''/VV^2r^) 

where C, c > are universal constants. 

3.2.3 Vectors on a sphere 

Another interesting case is when the vectors Xi, . . . lie on a common 
sphere. To keep the same normalization as in the previous cases we assume 
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that the sphere has the radius ^Jn. Then condition 03.51) becomes empty. Let 
1 < r < 2 and assume that the vectors are t\}r and let t\} = maxj<7v 
Let K > 1 and set ^ = ipK. Then Theorem 13.31 immediately gives that 



where C, c > are universal constants. 

4 The geometry of faces of random polytopes 

In this Section we discuss the geometry of random polytopes. Let A be an 
n X N matrix. We denote by /^^(A) (resp. K{A)) the convex hull (resp., 
the symmetric convex hull) of the columns of A. 

4.1 Neighborly polytopes 

For an integer 1 < m < n, a polytope is called m-neighborly if any set of less 
than m vertices is the vertex set of a face. In the symmetric setting, a cen- 
trally symmetric convex polytope is m-centrally-neighborly if any set of less 
than m vertices containing no-opposite pairs is the vertex set of a face. We 
refer the reader to the books [16] and [31] for classical details on neighborly 
polytopes. (Some new quantitative invariants related to neighborliness were 
recently developed in [22].) 

The relation between the problem of reconstruction and neighborly poly- 
topes was discovered in [TTj . 

Theorem 4.1. (ITl^ . Theorem 1) Let A be a n x N matrix, n < N. The 
following two assertions are equivalent. 

i) The polytope K{A) has 2N vertices and is m-centrally-neighborly. 

ii) Whenever y = Az has a solution z having at most m non-zero coordi- 
nates, z is the unique solution of the optimization problem (P); 

(P) min||t||£j, At = Az, 




(3.9) 



with probability larger than 
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We will also use the following result from [5] (which could be replaced by 
a similar result from [S]). 

Lemma 4.2. JS^ Assume that 52m{A/ < \pl — \. Then whenever y = Az 
has a solution z having at most m non-zero coordinates, z is the unique 
solution of the i\ minimization 'problem (P). 

We are now ready to state the main result on neighborly random poly- 
topes. 

Theorem 4.3. Let l<m<n<Nbe integers. Let 1 < r < 2. Let ■?/' > 1 
and A G (0, 1/2). Let Xi, . . . ^X^ he independent random vectors satisfying 
Hi{r, ip) with parameter ip and H2{X) with probability A. Let A be the n x N 
matrix with Xi, . . . , X^ as columns. Then, with probability larger than 



the polytopes K^{A) and K{A) are m-neighborly and m- centrally-neighborly , 
respectively, whenever 



where C, c > are universal constants. 

Observe that the probability is positive for n large enough provided that 



Proof. Theorem 13.31 and the definition of property Hi{r,ip) imply that for 
arbitrary 9' G (0, 1), and i^, K' > 1, setting ^ = tpK + K' , we have 



1 - 2X - C exp{-c^/ij^) 



A < 1/2. 




holds with probability larger than 



1 
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In view of Lemma [4.21 we look for m and 6' to ensure (52m (^/v^) < — 1- 
For instance, we let = (v/2 - l)/2 and note that (ES]) implies 

P I nia^lX^I > ( ^ + 1^ v^|<A. (4.2) 



/ /- \ 1/2 

So we take K' = ( + 1 ) and K = 1 which determines ^ = tpK + K' in 

terms of t/'. We shall use the fact that 1 < ^Z'?/' < C, where C is a universal 
constant. 

Now set mo = [dn/ijj'^log^^'^{C'ilj^N/n)] (for some new constants C',c' > 
0). It is clearly sufficient to prove that the polytopes K^{A) and K{A) are 
mo-neighborly and mo-centrally-neighborly, respectively. Thus adjusting the 
constants C, c' > and writing m for mo, we obtain 



Combining this with the choice of 6', passing from m to 2m and adjusting 
the constants again if necessary, we conclude that 6m (^'^^ < — 1 with 
probability larger than 




1 _ c"exp -c"v^log j= - 2A. 



The last estimate follows from fl4.ll) by applying 03.51) and (14.21) to the last 
two terms, respectively; and where C", c" > are again new constants. □ 



4.2 Examples 

We will now apply Theorem 14.31 in the three different settings introduced in 
the previous section. 

4.2.1 The Log-Concave Ensemble 

Applying Lemma 13.41 and bound (13. 6p we get the following: 
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Theorem 4.4. Let l<m<n<Nbe integers. Let Xi, . . . , Xj^ be in- 
dependent isotropic vectors with log-concave densities. This is for instance 
the case if Xi, . . . ,Xn are i.i.d. random vectors uniformly distributed on an 
isotropic convex body. Then, for any N < exp(cn'^^/^), with probability at 
least 1 — Cexp(— cn"^^/^), the polytopes K^{A) and K{A) are m-neighborly 
and m- centrally-neighborly, respectively, whenever 

m < cn/ log^ (CN/n), 

where C,c> are universal constants and c\ is given in Lemma \2.6[ 

Remark: It is known (|2j) that there is a universal constant such that the 
uniform probability measure on the ball {x G : Y17=i I^^T — 1} satisfies 
Hi{r,ilj) for 1 < r < 2 and satisfies H{2,ip) for r > 2. Of course, since 
it is log-concave, the concentration property H2 is also satisfied. Applying 
Theorem 14.31 to these examples, we get a better estimate of the level of 
neighborliness than in Theorem 14. 4[ We get now m ~ cn /log2/"(CAr/n) for 
1 < r < 2 and m ~ cn/ log(CA^/n) for 2 < r < 00. 

4.2.2 Matrices with independent ipr entries 

In a similar way as above. Lemma 13.51 and bound (13.81) imply the following 
theorem (note that its conclusion becomes empty if > exp(cn''/^/'?/'^'') and 

Theorem 4.5. Let A be a matrix with entries that are independent centered 
variance one random variables. Let 1 < r < 2 and assume that the ipr 
norms of the entries are bounded by some constant ip. Then, for any N < 
exp(cn''/^/'?/'^'"), with probability at least 1 — Cexp(— cn''/^/-?/'^''), the polytopes 
K^{A) and K{A) are m-neighborly and m- centrally-neighborly, respectively, 
whenever 1 < m < n satisfies 

m < cn/i)^\og^/'''{Ci>^N/n), 

where C, c > are universal constants. 

4.2.3 Vectors on a sphere 

Finally assume that the vectors are on a sphere of radius ^/n. From bound 
(13.91) we obtain: 
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Theorem 4.6. Let l<m<n<Nbe integers. Let 1 < r < 2. Let 

Xi, . . . ,Xn- be independent vectors on a sphere of radius ^/n and satisfying 
Hi{r, ip) for some parameter ip > 0. Let K > 1 and set ^ = ipK. Then, with 
probability at least 1 — C exp^—K^/n/ip^), the polytopes K~^{A) and K{A) 
are m-neighborly and m- centrally-neighborly , respectively, whenever 



where C,c> are universal constants. 

Remark: 1) For the matrix A with i.i.d. Gaussian A^(0, 1) entries (the case 
considered in Section [3.2.21 above when r = 2), it is known that with over- 
whelming probabihty, K{A) is m-centraUy-neighborly, whenever 1 < m < n 
satisfies 



where C,c > are universal constants, (see [12], [8], [27], [21]). The precise 
asymptotic dependence of m on n and has been well studied in [13] when 
n/N ^ 5 G (0, 1) and in [Mj when n/N 0. 




2) The Restricted Isometry Property was proved in [21] for matrices with 
independent rows (rather than columns), under a sub-gaussian hypothesis. It 
is worth noting that the corresponding result for matrices with independent 
isotropic sub-gaussian columns is not true in general. One can see it by 
considering the matrix with columns Xj = ^/2Si{eii, . . . ,eni), where 5i are 
independent random variables, P(5j = 1) = P(5j = 0) = 1/2 and eji are 
independent Bernoulli variables, independent of (5j's. The vectors are 
then isotropic and sub-gaussian, but P(Xj = 0) = 1/2. As a consequence, 
the concentration hypothesis and thus the RIP property are not satisfied. 

5 Main technical result 

In this Section, Xi, . . . , X^ € M" are independent ipr random vectors for 
some (fixed) < r < 2. Let 1 < m < N. We shall consider three quantities 
Am, Bm and Cm depending on Xi, . . . , X^r. Recall that Bm has been defined 
in (I32D as 



m < cn/^^log2/"(C^6X/n) 



m < en / \og{C N / n) , 





2 



1/2 
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and define the other two quantities as follows: 



Am = sup 

zeUm 



i<N 



, Cm = max 

i<N 



We clearly have 

I /l2 _ 7^2 I ^ --f2 
I m ml — m' 

Given a real number s, we will denote max(s, 0) by s+. 

The main purpose of this Section is to prove Theorem I3.2[ In fact we will 
prove a stronger technical result, Theorem 15. from which Theorem 13.21 will 
follow. 

Theorem 5.1. Let < r < 2. Let n > 1 and 1 < m < N be integers. Let 
Xi, . . . ,Xjsi G M" be independent ipr vectors with ip = maxj<7v ||Xj||^,,. For 
every 1 < m < N , ^ G (0, 1/4), and K > 1 one has 



F{Bl>me.x{B\CmB,2AdCl}) 

< (l + 3\ogm)exp( -2K''m'-^+'^/Hog — ] 
\ 6m J 



(5.1) 



with s = (1 — r) I and 



B = &/i)K nf-^'^ ( log ^ , , 

\ Om J 

where Co is an absolute constant and q = max{l, 1/r}. 

Remark: In fact we shall prove a stronger statement: with the notation of 
Theorem 15. H for every 1 < m < N, 6 ^ {0, 1/4), and K > 1, and for every 
< i < log2 m, one has 



2N\ 



l/r 



- 



P (S^ > max{s', CmB, 24 6 C^}) 



< (l + 2£)exp(-2ir^^logl^), (5.2) 



where 



+ m'^-^'^ log — 



6m J \ 6m J 
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C is an absolute constant and q = max{l, 1/r}. 

Before starting the proof of the theorem we show how it imphes Theorem 
13.21 stated in Section [3J 

5.1 Proof of Theorem [372] 

Fix Ki > 1 and let K > Ki he such that 

6m 

By Theorem 15.11 with r > 1, and the condition on m, 

2N\ 



< (1 + 31ogm) exp ^—2K^\^\og 

< exp (^--cKl^/m\og'^^ , 



- 



9m J 



where 
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B = Co^i^v^log^/'' — = CoijK.e^/^, 

Um 



and c and Co are absolute positive constants. Thus, if Cm < K2\fn for some 
K2, then 

< Cien{iljK^ + K2)\ 
where Ci is an absolute constant. This concludes the proof. □ 



5.2 Proof of Theorem 15.11 

We will prove the theorem in a stronger form (15. 2p . Then (15.11) follows by 
choosing < £ < log2 m to be the largest integer satisfying 

1 /, 2iV2'\''" ,,, / 2JV\'" 
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The proof will use the same construction as in [T], which however re- 
quires some modifications. For completeness and the reader's convenience 
we provide details of the argument. 

We require the following two lemmas proved in |Tjj with r = 1. Since the 
proofs for general r repeat the same arguments, we leave them for the reader. 

Lemma 5.2. Let < r < 2 and Xi, . . . , X^v G M" he independent ipr vectors 
with ip = maxj<7v ||Xj||^^. Let m < N , e,a & (0,1]- Let q = max{l, 1/r} 

12eAf\lA' 



and L > (2 log ^) . Th 



en 



P 



sup sup 



sup 



Fc{i,...,N} EcF z£Af(F,e,a) ~Z 

\F\<m. «S-fc/ 



1 



j^F\E 



> ip aLA^ 



< exp 1^-- m~("~i)+ 

Lemma 5.3. Let < r < 2 and Xi, . . . , X^v G MJ^ be independent ipr vectors 
with Tp = maxj<jv ||Xj||^^. Let 1 < k,m < N , e, a E {0,1], P > 0, and L > 0. 
Let B{m,P) denote the set of vectors x G with \ suppa;| < m and let B 
be a subset of B{m,P) of cardinality M. Then 



P 



sup sup 



sup 



Fc{i,...,N} x&B zeAf(F,e,a) 

\F\<k \ ' ' / jgi< 



< M 



j^F I 



> ijjapLAr, 



/GeNY 
VkT 



exp 



The following formula is well known and the proof is in its statement. 
Lemma 5.4. Let xi, . . . , xn G M", then 

if^j EC{1,...,N} ieE jeE'= 



We are now ready to start the proof of Theorem 15.11 
Proof of Theorem 15.11 As in [Ij, the construction splits into two cases. 
If £ = we set 

Mi9)= U MiE,0/A,l). 

BC{1,...JV} 

]E\=m 
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Otherwise, define positive integers aoi ^i, . . . , by := [m2~'^"'"^] — [m2^'^] 
for 1 < k < £ and •= [m2^^]. Observe that < m,2~^^^ for 1 < k < i, 
flo < m2^^ and Xlfc=o'^fc ~ R-^call that for E C {1, • • • , A^} we identify 

with the subspace of vectors in with coordinates supported by E. 

We consider (i + l)-tuples {{Eq^xo),. ■ ■ ,{Ei,xe)) where {Ek)o<k<e are 
mutually disjoint subsets of {1, . . . iV}, \Ei^\ < a^, e M^* for all < A; < ^. 
A (£ + l)-tuple {{Eq, xq), . . . , x^)) is said to be admissible if 



/ 2^ 

XkeJ\f{Ek,e2-'',d— I for l<A;<^,xoej\^(£;,^/4,l), 



E 

fe=0 



< 2. 



The set of all vectors x — Ylk=o^k associated to admissible {£+ l)-tuples 
{{Eo,xo), . . . , (Ei.xe)) will be denoted by M{9). 

We shall consider the details of the case £ > 0, the other case can be 
treated similarly. 

Fix ((Fq, Xq), . . . , {Fi, Xi)) to be admissible and let x — Yli=o ^ ^{9). 
Denote the coordinates of x by x{i), i < N, then 

\i<N i<N I i<N i^j 



So 



where 



\Ax\'-J2x{if\X, 



i<N 



D, = Y,{<i)Xi.<3)Xj)- 

Now we split Drc according to the structure of x. Namely we let 

e 

^-=E E {<i)Xi,x{j)Xj), 



(5.3) 



and 



fe=0 i&Fk 
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so that we have 



i<N 



\D'^+D:\<m + \D:\ 



We first estimate D'^. By Lemma [5.41 we have 



k=0 iJSFf, 



4^2H^'=i E E E {<^)x^,xm,) 

k=0 EcFk idE jeFk\E 



Thus 



i^;i<4E2-'^^-' E 

k=0 EcFk 



J2 E {<^)X^,xiJ)X,) 



< 



A:=0 



sup 

EcFk 



J2 E {4^)X^,x{J)X,) 

i&E j&Fk\E 



and using the fact that \Fk\ < ak for < A; < ^, we arrive at 



fc=0 



Fc{i,...,iV} E<ZF 
1-P'l<"fe 



a; 



We now set q = max{l, 1/r} and apply Lemma [5.21 to each summand in 
the sum above with the parameters 



ao,e = e/4,a = 1, and L = Ki^—j Ulog— 1 



for /c = and 

ak,e = 62^^, a 



„ -dL = A-y 4 log — 
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ioT 1 < k < £. By the union bound we obtain that the probabihty of the 
event 



^sup^Pil > ,A^k[{-:) (4 log—) 



ifc=i ^ 



is not larger than 



, 2m, 48eAr2^\ / 2m , 12eiV4* 

exp —A — ^ lo^ 



?\ " / 2m 
-J exp (^-X'^— log 



Therefore the probabihty of the event 

rA ( 48eiV2^^^^'' 



is not larger than 



(^^o\ I i^r 2m . 12eiV2' 
(l+^)exp -K"" -rf\o% 



2^ Qm 



where C\ is an absolute constant. 

We now pass to the estimate for D'^. which essentially follows the same 
lines. 

For every 1 < /c < £ we consider Mk{Q) = M'^iO) n 25^, where M'^iO) 
consists of all vectors of the form v — vq + ^s=fc+i ^s, where Vi^s {i = Q,k — 
!,...,£) have pairwise disjoint supports and 

voe U ^^{E,e/A,l), v,e [j Af(E,e2-',J-\ for s>k + l. 

EC{1,.-N} EC{1,.-N} \ ^ / 

\E\<aQ \E\<as 
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Then Aik{6) C and (similarly as in [T]) we can estimate the cardinality 



^ / s=k+l ^ / \ \ / s=k+l ^ 

2m, 3e4^A^\ /4m, 6e4'=A^\ 



Recalling that x = ^^^qX^ G A^(6') for some admissible {£ + l)-tuple 
((Fq, Xq), . . . , {Fi, xe)) and setting Gk = {0, k + 1, k + 2, . . . , i}, we observe 
that 



\D''\ 



< 2 sup sup sup 



Now we apply Lemma [573] to each summand A; = 1, . . . , with parameters 
_,/? = 2,^ = ^1,(0) and ^ = ^(^) ( 12 log J 



Using the union bound we obtain 

^' m \ 9-1/2 12eAr4'=V/''' 

fc=i 



F(^|D;i>2^A.irgy (12 log 

2^exp ( - 

fc=i ^ 

^ A / 6m 12eiV4^\ 6m 12eiV4^\ 



,'4m, 12e4'=A^ 2m, 3e4'=A^ ...12m, 12eA^4'= 



Thus 



P I sup 



, 6m 12eiV4^ 

< «exp —A — -lo^ 



2^ ^ ^m 
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where C2 is the an absolute constant. 
Since D^^D'^ + D'^, then 



P f sup^ \D,\ > 1 < (1 + 2^) exp l^-K^ log j , (5-4) 



where 



for some absolute constant C3 > 0. 



+ m^-V2(log^] 



Passing now to the approximation argument, pick an arbitrary z e S^~^ 
with I supp2;| < m. Define the following subsets of {1, ... , depending on 

z. Denote the coordinates of z by z{i) {i = 1, . . . , N). Let rii, . . . ,1%^ he such 
that > 1-2(^2)1 > ••• > |-2(^iv)|i so that z{ni) = for i > m (since 

I supp z\ <m). If £ = 0, we denote the support of z by £^0 and consider only 
this Eq. Otherwise we set 

Eo — {ni}i<i<Tn/2l 

and 

El = {ni}m/2<i<m, E2 = {ni}m/4<i<m/2, ■ ■ ■ , E^ = {ni}jn/2(<i<m/2^-'^- 

Recall that integers oq, oi, . . . , have been defined at the beginning of this 
proof. Then, clearly, 

oo = l^ol < = \Ek\ < m/2'= + 1 < m/2*^-^ for every 1 < k < £, 

and Yli=o o^i = nT-- Also observe that, since z e S^~^, then for every k > 1, 

f2k 



\PE,4oo<\z{ns)\<^-, 




where s = [m/2'']. 

For every k > 1 the vector Pg^z can be approximated by a vector from 
J\f (^E,,92-\ and the vector P^^z can be approximated by a vector 
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from M^Eq, 9/4, 1). Thus there exists x G M.{0), with a suitable representa- 
tion X = X]fc=o^^' ^'^'^^ that 

e e 
\z-x\^<Y, \Pe,^ -Xk?< 0\2-^ + ^ 2-2'=) < ^2 (0.4). 

fc=0 fc=l 

Moreover, x is chosen to have the same support as z, and thus w = z — x 
has the support | suppw| < m. 

It follows from the definitions of D, and A that 



D-^ = + {Aw, Ax) + (A^, Aw) - ^w{i) {x{i) + z{i)) \Xi 



i<N 



(here ^(2), x{i) and 2;(2) denote the coordinates of w, x and z, respectively). 
Thus 

I-D2I < l-Dxl + |v4t(7|(|ylx| + |y42:|) + \w\ \x + z\ niax|Xi|^ 
It follows that 

Bl= sup \D,\< sup \D,,\+29{Al + Cl)< sup \D.,\+29 {BI + 2CI) 

I supp z I <m 

Thus, by (15.41) and using again Am < + < B^ + Cm we obtain 

P ((1 - 29)Bl > A9Cl + Cml + S„.7) < (1 + 2^) exp (^-iT'^ ^ log —^^j ■ 

Since < 1/4, this implies 

2m , 12eA^2^ 



P (5^ > max{24^^C 6C„7, 67'}) < (1 + 2£) exp i-K^ — log ■ 



9m 

which completes the proof. □ 



5.3 Optimality of estimates 

We conclude this section by an example showing optimality, in a certain 
sense, of estimates in Theorem 13.21 We will limit ourselves to the ipi case, 
that is to r = 1. To this end we consider a special case when Xi = (Xij)^^^ 
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where Xij are i.i.d. symmetric exponential variables with variance one. We 
begin by showing an optimal estimate for A^- 

First, from [1] (Theorem 3.5) we have that for < exp{c^/n) and any 



where C, c > are numerical constants. In the other direction, we have the 
following 

Proposition 5.5. For any 1 < m < N and t > 1, 



where c> is an absolute constant. 

Before we prove this proposition let us explain its relevance to Theo- 
rem 13. 2[ Firstly, observe that the proposition implies that with probabil- 
ity bounded away from zero. Am > c{^/n + ^/m\og{2N / m)) . This shows 
that - except for allowing a change of absolute constants - one cannot ob- 
tain a better bound on A^ than (15. 5p . valid with overwhelming probability 
(i.e., with probability converging to one as n ^ oo). Secondly, assume 
that < exp(cA/ra). By taking t = cKs/n., we obtain that for large n, 
IP(^m > cK^/n) > exp{—cK^/n). We compare this with estimates for prob- 
abilities in (15. 5p . Namely, using Lemma [2.71 (noting that the density of Xj's 
is log-concave), we can see that for mlog^(2iV/m) < n, the theorem implies 
that P(Am > CK^Jn) < exp(— cfCy^). So in this range of m the upper 
and lower bounds on probability coincide up to numerical constants in the 
exponent. 

Regarding Theorem 13. 2[ again assume that < exp(cv^). Using again 
Lemma [2171 we get with overwhelming probability that for alH, |Xj| < C y/n. 
Now assume that for some m we have with overwhelming probability B"^^ < 
Cn. Then by the obvious bound A^ < + sup^g^^ J2i<N kiPl-^jP; with 
probability close to one we also have A^ < C" \fn. On the other hand, 
as noted above, P(Am > c{^/n \ ^Jm\og{2N / rn))) is bounded away from 
zero. Thus, c{y/n + y/mlog{2N/m)) < C" ^Jn^ which in turn implies that 
for n large enough we have mlog^(2iV/m) < Cn. This shows that the factor 
log^{2N/ 9m) in Theorem 13.21 is of the right order. 



K>1, 




(5.5) 
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Proof of Proposition 15.51 Since 



Am= sup sup y^^aiPjXjj, 

I supp a I <m 



by general tail estimates for linear combinations of exponential variables with 
vector valued coefficients (see e.g. Corollary 1 in [2lj), we get 



where 



and 



P(^m > c(EA„ + Via + tb)^ > c A e" 



a^ = sup sup ce'^Pj = 1 

I supp CK I <m 



6= sup sup max|aj/3j| 

I supp a I <m 



Therefore, it is enough to show that KA^ > c{V>^ + ^/mlog(2A^/^))- 
Obviously, > Ca/tz, since a single column of the matrix A has expected 
Euclidean norm of the order -^n. As for the other term, it is enough to 
consider the first row of our matrix. We have 



N 



VrnAm> sup y^aiYi, 



I I *=1 

I supp a I — m 

where to simplify the notation we set Yi = Xn. On the right hand side 
we actually have YllLi where Y* is such a rearrangement of Yi that 
> 1^2*1 — • ■ • — l^n I? which can be used to derive lower bounds on the 
expectation. We will however not rely on this representation, instead we will 
use a Sudakov type minoration principle for exponential variables proved in 
[28], Theorem 5.2.9, which we state here in a simplified version, adapted to 
our purposes. 

Lemma 5.6. Let Yi, . . . , Y^ be independent symmetric exponential variables 
with variance one. Consider T C of cardinality k and u > 1. If for any 
s,t e T, t s, 

Vu\t — s\ + u\\t — s\\oo > u, 

then E maXfgT ^ili ^j^j > cmin(M, log A;), where c > is an absolute con- 
stant. 
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In our case, T = {a G {0,-1,1}^: |suppQ;| < m}, so k > (^). Also, 
since \\t — s||oo > 1 for t, s & T, t ^ s, the condition of the lemma is trivially 
satisfied for any u > 1, in particular for u = logk. Thus, for m < N/2, 
we obtain ^/mKAm > logfc > cmlog{2N/m). On the other hand we have 
'^A^ > Cy/m, so for m > N/2 it is enough to adjust the constants. □ 
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